Mpeg-4 remote communication device

ABSTRACT

An MPEG-4 based multimedia system in connection with a service provider therefore, and a remote communication device therefore, is presented. The interactive content based remote communication unit simplifies and reduces the number of steps and buttons required for receiving interactive service by the use of MPEG-4 encoded data. Additionally, new ways are disclosed for the data processing at the service provider side. Further, an improved way of exchanging data between the service provider (broadcaster, cable provider) and the customer (viewer) is presented. Still further, there is disclosed how digital equipment (server, set-top box, remote control unit and display) involved in the data processing can be optimized for the task of controlling the available channel bandwidth and displaying the data.

FIELD OF THE INVENTION

[0001] The present invention concerns multimedia systems and moreparticularly, an MPEG-4 based multimedia system in connection with aservice provider therefor, and a content based remote communicationdevice therefor.

BACKGROUND OF THE INVENTION

[0002] It is well known that multimedia devices are versatile as to theability to process information of various types, e.g., audioprogramming, television programming, movies, computer games, internetcommunications, etc., and provide the processed information to a user.However, the processed information may only appeal to a single user,e.g., the user who selected the programming. Other users who are presentwhen the processed information is presented (e.g., displayed or outputthrough an audio system) might prefer other programming or, for example,they might prefer to obtain more information about a particular aspectof the present programming. In addition, it is well known that thecontrol of multimedia devices can be difficult. Most current remotecontrol units play a passive role as they only transmit one-waycommands, e.g., to a television receiver, and hence represent arelatively primitive interface between a viewer and a device. Typicalmultimedia applications include interactive services which require alarge number of buttons on remote control units. This is cumbersome andconfusing especially because limited space is available on the devicesurfaces for the buttons/keypad. As the number of multimedia devicesincrease in the future, the number of corresponding features will alsoincrease making the situation even worse.

[0003] Therefore, various companies have come up with new solutions tomeet the challenge of controlling modern multimedia equipment. One ofthese solutions is the PRONTO™ intelligent remote control made by thePhilips™ company headquartered in the Netherlands. This remote controlunit is a universal learning device, which CONFIRMATION COPY includes alarge touch-screen liquid crystal display (LCD), a virtual keyboarddisplayed on the LCD, and a two-way infrared (IR) transceiver. The IRtransceiver is used to learn codes from other remote control units(RCUs). The Philips™ remote control unit is considered to be anintelligent device because of its adaptive number of virtual buttons. Asa result, the user only sees what he/she needs to see for performing adesired function.

[0004] The Samsung™ company headquartered in South Korea has taken afurther step and makes a two-way remote control unit named IDEO™ that isequipped with a small high-resolution LCD display. The Samsung™ remotecontrol allows for the reception and viewing of television signals onthe display in addition to the normal functions of a remote controlunit. Thus, while viewing one program on the television, the user canscan, preview, view or select other programs/channels. It should benoted that the source of the video signal shown on the remote controlunit display is an external device, e.g., a transmitter included in thetelevision receiver, and such an arrangement requires a second tuner.However, the virtual buttons are internally produced by means ofappropriate software and/or hardware.

[0005] U.S. Pat. Nos. 6,671,225, 5,861,906 and 5,657,072 show thebi-directional transmission of digital information between a mediaserver and a set-top box which is connected to a television receiver.

[0006] U.S. Pat. No. 6,002,450 shows a two way remote control devicewith an LCD display providing a visual display of selected informationsuch as an advertisement.

[0007] U.S. Pat. No. 6,020,881 shows a remote control device having agraphical user interface and has objects which can be selected by theuser.

[0008] U.S. Pat. No. 6,070,167 shows a hierarchical system for theobject based audiovisual tagging of images for, inter alia,manipulation.

[0009] U.S. Pat. No. 6,097,441 of Allport shows a system using two ormore cooperating but physically independent displays. This patentappears in some ways to be similar to the Samsung™ remote control devicediscussed above.

[0010] U.S. Pat. No. 6,127,941 of Van Ryzin shows a two-way wirelessremote control unit including a graphical user interface for controllingvarious multimedia devices. This and the Philips™ device discussed abovehave similar remote control features but do not operate in the samemanner concerning signal transmission.

[0011] U.S. Pat. No. 6,130,726 shows a remote control device with adisplay for showing a program guide.

[0012] U.S. publication No. US2002/0016766 shows the bi-directionaldistribution of digital content with a service provider.

SUMMARY OF THE INVENTION

[0013] As technology rapidly advances, there is a need to furtherimprove both the ability to access multimedia content and theabove-mentioned remote control concepts by taking advantage of newtechnologies. The present inventions are based on two different dataencoding formats, e.g., the Moving Pictures Expert Group's MPEG-2 andMPEG-4 multimedia standards. MPEG-4 is a content-based encoding/decodingprocess and is not block-oriented as is MPEG-2. Hence, MPEG-4 allows forthe ability to interact with objects that make up the audio-visualscene. These objects can be audio, visual and audio-visual objects,which can be natural or synthetic, i.e., they can be recorded with acamera, a microphone, or generated by a computer. The data streams ofMPEG-4 contain object and scene descriptors that provide configurationand other information for the streams related to interactiveaudio-visual objects. Thus, the MPEG-4 standard also allows for improvedclient-server interactivity, which can be realized in the form ofdownstream and return stream communication channels. This is usefulbecause a downstream data stream may require information to betransmitted upstream from the receiver to the sender, for example, ine-commerce or interactive television.

[0014] An MPEG-4 data stream can be embedded and transmitted in theMPEG-2 data stream, and can subsequently be separated from the MPEG-2data stream, i.e., at the receiver side.

[0015] The present invention takes advantage of the above-mentionedfeatures of the two data encoding formats MPEG-2 and MPEG-4, anddiscloses an interactive remote communication unit having an MPEG-4decoder. The disclosed interactive remote communication unit simplifiesand reduces the number of steps required for receiving interactiveservice. Such a remote communication device also reduces the number ofbuttons required for operation of the remote since a large number can beconfusing to the user. The present invention also discloses new ways fordata processing at the provider side.

[0016] Interactive digital television services are now considered a keyelement to introduce digital television services and to encouragecustomers to invest in new digital television appliances. The presentinvention discloses improving of the way of exchanging data between theservice provider, e.g., broadcaster, cable company, Internet serviceprovider, and the customer (viewer). The present invention alsodiscloses how digital equipment (server, set-top box (STB), remotecommunication unit and display) involved in the data processing can bemodified to optimize the task of controlling the available channelbandwidth and the displaying of data.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017]FIG. 1 shows a service provider arrangement for interactivedigital television services.

[0018]FIG. 2 is a block diagram that shows how the service providerprepares the titles and the additional optional audio-visual objects ofdifferent programs.

[0019]FIG. 3 shows a block diagram of the customer side of FIG. 2.

[0020]FIG. 4 shows a flow chart of the processing algorithm for theset-top box of FIG. 3.

[0021]FIG. 5 is a diagram of a representative multimedia system.

[0022]FIG. 6 shows a block diagram of the remote communication unit ofFIG. 5.

[0023]FIG. 7 shows a block diagram of the service provider of FIG. 2using statistical gathering of information.

[0024]FIG. 8 shows a flow chart of the operation of a remotecommunication unit of FIGS. 5 and 6.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

[0025] In FIG. 1, a service provider for interactive digital televisionservices 10 often has at least two kinds of data available for (N)customers 12, 14, 16 18. The data of a main program, e.g., a film,comprises a main event. Additional interactive audio-visual informationrelated to the main program, e.g., short films, are designated herein as“additional audio-visual objects” or simply “additional objects”.Because channel bandwidth is limited, the service provider is obliged tooptimize the amount of data transmitted per channel. Since theadditional objects contain optional information, only a portion of thecustomers 12, 14, 16, 18 may be interested in receiving the additionalobjects. Therefore, it would be a waste of channel capacity if theadditional objects were transmitted all of the time.

[0026] To solve this problem, the present invention, inter alia,discloses that service provider 10 first transmit “the titles” of theadditional objects to all the customers and then transmit the actualobjects only to the interested customers. A title is defined herein as ashort text or an icon that indicates the availability of an additionalobject. Service provider 10 transmits such a title with itscorresponding object code, i.e. the code of the additional object thatcan be downloaded by customers 12, 14, 16, 18. The title and later theadditional object are received and processed by a receiving device,herein typically an integrated receiver-decoder set-top box or simplySTB (1) to STB (N) 20, 22, 24, 26.

[0027] Another problem is that the customers are confronted with a largeamount of data and options that can be difficult to manage. For example,when the additional objects are simultaneously displayed on the sametelevision screen as the main program, they would cover a part of themain picture. There are also other problems that might arise when morepeople watch the main program and there would be a plurality of displaysoverlaid on the main picture. Additionally, usually not all of theviewers would want to view the additional objects.

[0028] To solve these problems, inter alia, a two-way RCU 28, 30, 32,34, 36, 38, 40, 42 with respective displays 44, 46, 48, 50, 52, 54, 56,58, which may, for example, be touch-screen displays, is disclosed withbi-directional communication between the STBs and RCUs. When STB 20, 22,24, 26 receives a program including a title, STBs 20, 22, 24, 26separate the title from the main program and transmit the title torespective RCU 28, 30, 32, 34, 36, 38, 40, 42. The title is transmittedto the RCUs via a communications medium, such as a wired connection(e.g., a bus or network connection) or a wireless connection (e.g., anRF or infrared wireless connection, such as a wireless network or homenetwork connection) and is received via a signal receiving meansincluded in the RCU, e.g., included in transceiver 88 shown in FIG. 3.RCUs 28, 30, 32, 34, 36, 38, 40, 42 then display the title on itsrespective display 44, 46, 48, 50, 52, 54, 56, 58 and no title appearson the main television display. One touch on a title appearing on thetouch-screen display or, for example, activation of an appropriatebutton or switch on the RCU, suffices to activate transmission of amessage to the respective STB for the purpose of ordering the additionalobject. This is accomplished by the respective RCU 28, 30, 32, 34, 36,38, 40, 42 transmitting a message back to respective STB 20, 22, 24, 26via a second transmission medium, e.g. wired or wireless, that may bethe same or different as the first medium used to receive the title fromthe STB (e.g., messages could be sent both directions via a wireless RFcommunication channel or, alternatively, the title could be sent to theRCUs via an RF communication channel (or via a wired home networkconnection) and messages could be sent from the RCU to the STB via aninfrared wireless protocol. The transmission from the RCU to the STBoccurs via a transmission means that may, e.g., be included intransceiver 88 shown in FIG. 3. A transmission means as referred toherein may also be viewed as including a portion of the touch screendisplay or RCU control system that when activated by a user enables orcauses generation and transmission of such a message from the RCU to theSTB. This message from the RCU to the STB is designated herein as a“title request message”. STB 20, 22, 24, 26 processes the “title requestmessage” and transmits the code of the desired object back to serviceprovider 10 via a return channel. The return channel may or may not bethe same medium as the downstream channel. For example, the downstreamchannel can be a satellite channel and the return channel can be, e.g.,a satellite channel, an Internet channel, a cable channel etc.

[0029] As shown in FIG. 1, a customer might have more than one RCU, forexample customer (1) has three RCUs, specifically RCU(1-1) 28, RCU(1-2)30 and RCU(1-3)

[0030] 32. As will be discussed later according to aspects of thepresent invention, every RCU 28, 30, 32, 34, 36, 38, 40, 42 has its ownidentification code and can be individually addressed by its respectiveSTB 20, 22, 24, 26. As a result, an additional object can be displayedon RCU(1-1) 28 while RCU(1-2) 30 completely ignores that object. Whenmore than one RCUs are available to the customer, then these RCUs alsocommunicate with each other to prevent simultaneous data transmission tothe respective STB 20, 22, 24, 26.

[0031]FIG. 2 is a block diagram that shows how service provider 10prepares the titles and the additional optional audio-visual objects ofthe different programs. The current main programs, e.g., film (A) andfilm (B), are stored in MPEG-2 storage medium 60, and live program (C)and live program (D) are provided at 62. It is assumed for thesimplicity of discussion that only film (A) and live program (D) areeach provided with one additional object but obviously any and all itemscan be provided with a plurality of additional objects.

[0032] The uncompressed audio-visual digital signal of the main liveprogram (D) is coupled to and processed by an encoder for encoding datain a first format, e.g., MPEG-2 encoder 64, in a manner known in theart. The compressed MPEG-2 packetized streams at the output of MPEG-2encoder 64 are applied to a multiplexer 66. Similarly, the MPEG-2packetized streams of film (A) are transferred from storage medium 60directly to multiplexer 66 since film (A) is already stored in MPEG-2compressed encoded form to reduce memory space. Thus, the describedMPEG-2-related signal handling and processing features comprise anexemplary embodiment for providing or generating one or more main eventsor programs in the form of a main data stream.

[0033] A “title developer” 68 produces the titles for the additionalobjects, which will be provided along with the main programs (A) and(D). Service provider 10 distributes the generated titles to all thecustomers from customer(1) 12 through customer(N) 18. The titles areproduced by an encoder encoding data in a second format, e.g., in anMPEG-4 encoded form at 68, by using software tools known to thoseskilled in the art. According to the exemplary embodiment, a title notonly contains a short description about the object to be downloaded, butit also contains an identification code of this object. The producedtitle is applied as an MPEG-4 data stream to multiplexer 66, where it isembedded in the corresponding MPEG-2 packetized streams of, e.g., film(A) and live program (D). The described MPEG-4-related signal handlingand processing features comprise an exemplary embodiment for generatingor providing a data stream in addition to the main data stream, i.e., anadditional data stream.

[0034] Finally, some of the service provider's private data can also beapplied to multiplexer 66. Service provider 10 determines the content ofthe private data, which among other things, can include the address of aserver 72. This address can be used by STB 20, 22, 24, 26 to send back amessage to server 72 via a return channel, as will be discussed later.Thus, the resulting data stream at the output of multiplexer 66 is anMPEG-2 transport stream that can include, e.g., film (A), an MPEG-4embedded title of film (A), a live program (D), an MPEG-4 embedded titleof live program (D), and private data from service provider 10.

[0035] Additionally, it is also desirable that service provider 10 onlytransmit a title of an object while the corresponding event is stillbeing transmitted, for example, as long as film (A) is beingtransmitted. This is because service provider 10 initiates new titlesand new objects for subsequent events.

[0036] The data stream is coupled to a transmitter 74, where it ismodulated and transmitted to customers 12, 14, 16, 18. Prior to theactual data transmission, there are some required modulation and signalprocessing steps that are not relevant to the present invention, andtherefore, except for the immediate following listing, will not befurther discussed herein. The modulation and signal processing steps fortransmission will depend upon whether the signal is transmitted viasatellite, cable or terrestrial, with the contemporary technicalstandards being, e.g., the DVB-S (Digital Video Broadcasting-Satellite)standard, the DVB-C (Digital Video Broadcasting-Cable) standard, theDVB-T (Digital Video Broadcasting-Terrestrial) standard, the ATSC(Advanced Television Systems Committee) standard, and the DAB (DigitalAudio Broadcasting) standard.

[0037] In FIG. 3, the customer side 12, 14, 16, 18 of FIG. 2 is shown,e.g., customer (1) 12 with STB(1) 20 and three RCUs, RCU(1-1) 28,RCU(1-2) 30 and RCU(1-3) 32. Every RCU 28, 30, 32 has a uniqueidentification code that is stored in a memory in the corresponding RCU28, 30, 32. STB(1) 20, RCU(1-1) 28, and/or RCU(1-2) 30 and RCU(1-3) 32together comprise a multimedia system, in which the three identificationcodes of the RCU 28, 30, 32 are known and stored in a memory 80 of STB20. As used herein, a remote communication unit, or an RCU, isenvisioned as encompassing various forms and functions. For example, anRCU may provide for communicating with a service provider via a STB,e.g., for interactive services, such that the STB links the RCU to theservice provider. An RCU may provide for displaying content receivedfrom an STB, e.g., in the form of MPEG-4 data that the STB separatedfrom a combined MPEG-2/MPEG-4 data stream received by the STB. An RCUmay provide for controlling the STB, for example, by providing controlfunctions typically provided in a remote control. An RCU may provide forboth interactivity and STB control or may provide minimal or no STBcontrol functions such that the reception and transmission ofinformation via the STB is transparent to a user of an RCU. An RCU maybe handheld, e.g., similar to a typical remote control or a personaldigital assistant (PDA), or an RCU may be a larger device having alarger display, e.g., similar to a laptop computer.

[0038] Referring again to FIG. 3, in STB(1) 20, the received anddemodulated signal is an MPEG-2 data stream that is applied to an inputof demultiplexer 74. The exemplary data stream contains, as statedabove, e.g., film (A), an MPEG-4 embedded title of film (A), liveprogram (D), an MPEG-4 embedded title of live program (D) and privatedata of the service provider 10.

[0039] Assume for the purposes of the following discussion, that theviewer for STB(1) 12 selects to watch film (A). As a result,demultiplexer 74 demultiplexes the audio and video packetized streams offilm (A) and ignores the streams of live program (D). The output ofdemultiplexer 74 is applied to an MPEG-2 decoder 76, which decodes thesignal into the corresponding audio and video signals and hence film (A)appears on the color television (CTV) (not shown in FIG. 3).Demultiplexer 74 also detects and extracts the MPEG-4 embedded datastream from the MPEG-2 data stream. Particularly, the MPEG-4 data streamcontains the title of the additional object related to film (A). At thispoint and before transmitting the MPEG-4 data to RCU 28, 30, 32,microcontroller 78 creates a table designated herein as the “additionalobject table”. This table provides the information needed for handlingthe requested object in a proper interactive way between the viewer andservice provider 10.

[0040] The “additional object table” table can contain the followingdata:

[0041] 1—The object identification code that the service providertransmits with the corresponding title.

[0042] 2—Film (A) or event identification code which is included in theMPEG-2 transport stream.

[0043] 3—The current program (channel) identification code which isincluded in the MPEG-2 transport stream.

[0044] 4—The address information about where to send back messages tothe service provider via a return channel. This address might beincluded in the private data or in the service information provided bythe service provider.

[0045] 5—The RCU identification code(s) of the interested viewer(s).This is not known at the moment.

[0046] All the information but the RCU identification code are known andcan be readily stored in the “additional object table” in memory 80 ofSTB 20. Under the control of microcontroller 78, the threeidentification codes of the available RCUs and the separated MPEG-4title are transmitted via an RF-transceiver 82 to all of the remotecommunication units RCU(1-1) 28, RCU(1-2) 30 and RCU(1-3) 32. The RCUsthat receive and identify their own identification codes, here RCU (1-1)28, RCU (1-2) 30 and RCU (1-3) 32, decode the message using theirrespective microcontrollers (not shown in FIG. 3) and respective MPEG-4decoders 84. RCUs 28, 30, 32 also store the received objectidentification codes and display the title on their respectivetouch-screen displays 86. Thus, viewers can watch film (A) on the CTVand see the title, i.e. the received title of the additional object offilm (A) on the display 86 of their RCUs.

[0047] Assume that only the viewer with RCU(1-1) 28 chooses thedisplayed title by touch pressing the title on display 86 while theother viewers were not interested. In accordance with aspects of thepresent invention, a “title request message” is only generated by theRCU of the interested viewer, here RCU(1-1) 28. The “title requestmessage” includes the object identification code of the selected titleand the identification code of RCU(1-1) 28. The RCU transmits the “titlerequest message” to RF-transceiver 82 of STB 20 via its ownRF-transceiver 88. With the aid of the received object identificationcode, microcontroller 78 of STB 20 recognizes the “additional objecttable” and adds the received RCU identification code to the table, i.e.adds exactly to the same group of codes for the desired object.

[0048] The “additional object table” is now complete. Microcontroller 78then compares the code of the current received event in the MPEG-2transport stream, along with film (A), with the code of film (A) beingin the “additional object table”. Both codes are identical when film (A)is still the received event. Microcontroller 78 then sends an “objectrequest message” to the service provider 10 via the Internet or anotherreturn channel, e.g. the same medium as the downstream channel, usingmodem 90.

[0049] The server address of the service provider 10 is included in theprivate data as discussed before. The “object request message” includesthe object identification code and the set-top address, e.g. theInternet address.

[0050] Since the data exchange between the RCU and the STB takes placelocally, i.e., at home, the transmission protocol can be freelydetermined by the producer of STB 20. For example, a protocol similar toa typical remote control protocol might be used, e.g., for controlinformation, and/or other protocols such as a high-speed data networkcommunication protocol might be used for program content and/or controlinformation. In addition to the exemplary embodiment described hereinutilizing wireless RF communication between the RCU and STB, wiredcommunications may also be used rather than, or in addition to wirelesscommunication. If a network protocol is used, the RCU and STB may alsocommunicate with other devices, e.g. other multimedia devices such as anaudio system, a video program source (VCR, DVD, etc.), in addition tocommunicating between the STB and RCU. On the other hand, thetransmission protocol used between the STB and service provider 10depends on the medium used for the return channel and is not part of thepresent invention.

[0051] Of course, other viewers may also be interested in receiving thesame object, for example, in FIG. 1 the viewer with RCU(3-2) 38 atcustomer (3), and the viewer with RCU(N-2) 58 at customer (N). Atservice provider 10, (see FIG. 2), the object code and the STB addressesof all the interested viewers are collected and processed in server 72.Server 72 fetches the required object from the MPEG-4 storage medium 70by means of the received object code. In storage medium 70, this data isstored in MPEG-4 encoded form to reduce memory space. Server 72transmits the object, including its identification code, to thecorresponding customers via the downstream channel.

[0052] It should be noted that users of the various RCUs can viewobjects in a “time shift mode”. Assume that there are at least two RCUs(A) and (B) communicating with the same STB. The STB transmits a titleto both RCUs (A) and (B) with the title being simultaneously displayedon the RCU displays of RCUs (A) and (B). A viewer with RCU (A) canselect (press) the title on the display of RCU (A) at, e.g., 10:00o'clock, receives the object at 10:01 and starts to watch the requestedobject. A viewer with RCU (B) selects (presses) the same title on thedisplay of RCU (B) at 10:05 o'clock, receives the same object at 10:06and starts to watch the requested object. Assume further that theduration of the selected object is longer than the five minute timedifference between the starting time of watching the same object on therespective RCUs. In such a case, the viewer with RCU (B) watches theobject in a time shift mode relative to the viewer of RCU (A).

[0053] In FIG. 3, microcontroller 78 in STB 20 compares the MPEG-4received object code with the object code in the “additional objecttable” to get the corresponding RCU identification code of theinterested viewer, here RCU(1-1) 28. Microcontroller 78 then transmitsthe RCU identification code and the actual object via RF-transceiver 82to the RCU. Since only the identification code of RCU(1-1) 28 matchesthe received identification code, RCU(1-1) 28 decodes and displays theobject related to film (A) on its display 86, with RCU(1-2) 30 andRCU(1-3) 32 ignoring the received message. The displayed object isrelated to film (A) which is displayed on the CTV.

[0054] Thus, from the above, it should be clear that at the beginning ofa new offer from service provider 10, all the viewers receive a smallamount of additional data, namely the title. Later, only the interestedviewers receive the larger amount of additional data of an actual objectwhich can be a video signal with sound, e.g., a short subject film.Meanwhile, film (A) is being watched on the CTV without any disturbancecaused by extra windows that would cover the main picture, as would bethe case of, e.g., a picture-in-picture feature commonly known as PIP,which also requires a second tuner. As shown in FIG. 2, the title of theadditional object is embedded in the data stream of film (A), so thatwhen film (A) ends, the transmission of its corresponding title alsoends.

[0055] At the end of film (A), when the transmission of the title stops,a customer can be in one of the following situations:

[0056] A) An uninterested viewer who simply ignored the displayed titleor has already deleted it. This viewer is not effected at the end of thefilm.

[0057] B) A viewer has already downloaded the object of interest. Thisviewer is also not effected.

[0058] C) A viewer is in the middle of downloading the object. Thisviewer is also not effected because, in the present embodiment, server72 of service provider 10 continues to deliver the complete object.

[0059] D) A viewer requested an object after the film has finished. Ifthis viewer tries to download the object by pressing the title, a “titlerequest message” is generated and transmitted to STB 20 as explainedabove. Microcontroller 78 checks the present status of Film (A). Thestatus is either “event is currently running” or “event has alreadyfinished”. The information about the present status can be determined bycomparing the corresponding data in the MPEG-2 transport stream and theevent code of (film (A) stored in the “additional object table”. WhenSTB 20 compares both codes, then the codes will not be identical and thecurrent event status will be “event has already finished”. This statusis however true if the viewer did not change the program while watchingthe film. It is therefore necessary to check the status of the currentprogram. Microcontroller 78 compares the current program code with theprogram code stored in the “additional object table”. Since the viewerdid not change the program, both codes will be identical. The result ofthe comparison is that the film has definitely finished.

[0060] As a result, microcontroller 78 returns a “not available message”to the corresponding RCU and this message is displayed on respectivedisplay 86. The “not available message” is useful because it informs theviewer that the displayed title is superfluous, i.e, not needed anymore, and should be deleted.

[0061] Since the number of the remaining displayed titles can grow withtime, it would be helpful for the viewer if the deletion of theunnecessary titles would take place in a more efficient and automaticmanner. It is therefore desirable that the deletion of the titles can beachieved by one or more of the following possibilities:

[0062] 1) The viewer can delete a title any time in a conventional wayby highlighting a title and choosing delete from a menu.

[0063] 2) The viewer can delete a title after receiving a “not availablemessage” as explained above.

[0064] 3) The viewer can delete all superfluous titles by choosing theoption “refresh display” from a menu. The refresh display commandautomatically generates a “titles request message” for every displayedtitle. All of the “title request messages” are successively transmittedto STB 20 and checked for their availability. Depending on the result,STB microcontroller 78 sends a delete command to the corresponding RCU.By doing so, all superfluous titles are automatically deleted inseriatim.

[0065] 4) Full automatic deletion of superfluous titles can also beachieved by every time a user presses a title of interest, that a“refresh display” command is automatically activated.

[0066] 5) A viewer has changed the program before pressing the displayedtitle, i.e. although the event film (A) is not yet finished, anotherprogram has been selected. In such an event, the RCU generates a “titlerequest message” and transmits it to STB 20. STB 20 compares the currentevent code, (not film (A)), with the film (A) code in the “additionalobject table”, where both codes are not identical. When this happens,microcontroller 78 compares the current program code with the programcode in the “additional object table”. Since the viewer did change theprogram, the codes will not be identical. In this case, microcontroller78 sends a message “Switch back to the desired program (Yes/No)?”, whichwill be displayed on display 86 of the RCU. If the viewer chooses the“yes” option, then STB 20 automatically switches back to the program offilm (A). This is possible since the program code is stored in the“additional object table” as explained above. Then, microcontroller 78checks the current event status to find out if film (A) is stillrunning. If both codes are identical, then microcontroller 78 transmitsthe desired object code and STB 20 address to service provider 10. Ifthe program codes are different, then a “not available” message istransmitted to the corresponding RCU. On the other hand, if the viewerchooses the “No” option, then STB 20 ignores the “title requestmessage”.

[0067] The processing algorithm for the STB is shown in the flow chartof FIG. 4. The flow chart of FIG. 4 contains only the main stepsrequired for understanding aspects of the present invention.

[0068] Starting at STB block 400, a determination is made at 402 whetherthere was a transceiver interrupt. If the determination is NO, then adecision is made at 404 as to whether there was a multiplexer or modeminterrupt. If NO, then go back to 402. If YES, then there is adetermination at 406 whether it is an MPEG-4 title or an object. If itis an object, then the object is identified at 408, the RCUidentification code is gotten from the additional object table at 410,the RCU identification code is sent at 412, the object identificationcode is sent at 414, the object is sent to the RCU at 416, and a returnis made to 402.

[0069] If the determination at 406 is that the data is an MPEG-4 title,then an additional object table is created at 418, all RCUidentification codes are sent at 420, the title and object codes aresent at 422, and a return is made to 402.

[0070] If the determination at 402 is YES, that it is a transceiverinterrupt, then a determination is made at 424 whether it is an internalor external object. If the determination is that the data is an internalobject, then the non-MPEG data is decoded at 426, and the internalobject subroutine is invoked at 428 with a return to 402.

[0071] If the determination at 424 is that the data is an externalobject, then a title request message is provided at 430, the RCUidentification code is added to the object property list at 432, and adetermination is made at 434 whether the event codes are identical. Ifthe determination is YES, then an object request message is sent to theservice provider at 436 with a subsequent return to 402.

[0072] If the determination at 434 is NO, that the event codes are notidentical, then a determination is made at 438 whether the program codesare identical. If the decision is YES, then a “not available” message issent to the RCU at 440 with a subsequent return to 402.

[0073] If the determination at 438 is NO, that the program codes are notidentical, then the message “switch back to desired program (yes/no)” issent to the RCU at 442 and a determination is made at 444 depending uponthe received response to the message at 442. If the response is NO, thenthe title request message is neglected at 446 with a return to 402. Ifthe returned response is YES, then there is a switch back at 448 for adecision at 450 of whether the event codes are identical. If thedetermination at 450 is YES, then an object request message is sent tothe service provider at 452 with a return to 402. If the determinationis NO that the event codes are not identical, then a “not available”message is sent at 454 and a return is made to 402.

[0074] The system of FIG. 5 includes a CTV set 100, an STB 102 and atwo-way RCU 104 with the Internet being used as the return channel.Therefore, STB 102 is connected via a modem 106 to an Internet site. RCU104 includes an RF-transceiver 108 with an antenna for performing theappropriate processing operations for transmitting and receiving radiofrequency (RF) signals to and from STB 102, in a manner known in theart. Additionally, RCU 104 and STB 102 might include an infrared (IR)transmitter/transceiver (not shown) in place of the RF transceiversdiscussed herein, for wireless communication between an RCU and STB, ina manner known in the art. Another alternative is that the STB and RCUcould be connected by, and communicating via, a wired or wirelessnetwork to which other devices, e.g., one or more of a home audiosystem, personal computer, display device, DVD player, etc., might alsobe connected. Also as described herein, a system may include multipleRCUs that can communicate with the STB, with the service provider andwith each other via one or more of the described communication mediums.

[0075] Further RCU 104 is provided, for example, with a touch-screengraphical user interface, for example, an LCD 110, which is alsosuitable for displaying video. In one mode of operation, RCU 104operates in a conventional way, i.e. the displayed data representcommands as issued by the user to control various functions, usinginternally produced virtual buttons/icons appearing on the touch-screendisplay. STB 102 includes a second RF-transceiver 112 with an antenna.Second RF-transceiver 112 is substantially identical to firstRF-transceiver 108 of the RCU 104 and performs the same functions oftransmitting and receiving information carrying signals.

[0076] As can be seen on display 110 of RCU 104, in addition to theinternally produced icons, there are an arbitrary number of titles,which are designated here as Title-1, Title-2 and Title-3. All or a partof the titles may relate to the current event, the rest can besuperfluous. The user can download the corresponding object(s) in aninteractive way by pressing a title on the touch-screen display. As aresult, the selected title might be highlighted, as is shown in FIG. 5for Title-1.

[0077]FIG. 6 shows a simplified block diagram that represents the signalprocessing in RCU 104 according to aspects of the present invention. Itis understood that various other components are included in the RCU tocarry out different operations. However, for simplicity, theillustration of these components is omitted as being non-essential tothe understanding of the present invention. It should also be noted thatthe IR transceiver in the RCU and the IR transceiver in the STB areoptional and could be removed if RF transceivers 108, 112 are used tosend conventional control commands. Alternately, the RF transceivers canbe removed if the IR transceivers used also transmit MPEG data.

[0078] In FIG. 6, the received and demodulated MPEG-4 data streams atthe output of the RF-transceiver are applied to a microcontroller 114and an MPEG-4 decoder 116. The decoding process requires a memory 118for data storage. The decoded MPEG-4 digital audio and digital videosignals are further processed in an audio-video processing circuit 120in a manner known in the art, which further functions in a manner knownin the art, as an interface to a LCD touch-screen display 122 andloudspeakers 124, which can be used when display 122 displays arequested video object. RCU 104 also receives and decodes non-MPEG datafrom STB 102.

[0079] Up to this point, the object request messages and the actualadditional objects are transmitted mainly via, e.g., the Internet. Theadvantage of using the Internet as a return channel is to reduce theamount of data payload in the main transmission channel, e.g., asatellite channel. The disadvantage of this approach is that theadditional information has to be sent as many times as the number ofinterested customers. Thus, it is also an aspect of the presentinvention to consider return channels other than the Internet.

[0080] The return channels can be the same medium as the downstreamchannel, e.g., using cable, satellite or terrestrial channels. Thesereturn channels are defined in the standards discussed above. In theevent that the service provider uses the same medium for the downstreamand the return stream, then the amount of the main and the additionaldata should be optimized for that particular return channel. Inaccordance with aspects of the present invention, the titles of theadditional objects are transmitted in the same manner independent of themedium of the return channel, i.e. embedded as MPEG-4 encoded data inthe MPEG-2 packetized data streams of the main program. However, thedesired object codes of the different customers can be transmitted backto the service provider via the same medium as the downstream data orany other appropriate return channel.

[0081] On the service provider's side, the received messages arecontinuously sorted and counted separately for every object code, tocalculate the statistical distribution of customers' requests. Theresult will show how many customers are interested in which object. Thisresult is then used by, e.g., a statistical multiplexer 126 shown inFIG. 7, before sending the actual additional objects. It is of course tobe understood that the exemplary embodiment uses a stastical multiplexerbut that there are other equivalent ways of developing a statisticaldistribution, e.g., in the software of multiplexer 66, or otherequivalent devices, all being commonly available. As a result, the morecustomers that are interested in receiving one of the additionalaudio-video objects, the more data space for this particular object ismade available. The statistical distribution of the incoming messages isalso valuable information about the present interest of the viewers andthis is performed by server 128. With the aid of this information, theservice provider can improve the quality of the additional objects andreduce/add to the bandwidth used. In this embodiment, bi-directionalcommunication is performed by transmitter/receiver 130 using thedownstream channel for the return channel. Except for the statisticalgathering functions shown in FIG. 7, FIG. 7 includes many of the samemembers as FIG. 2 and such members are similarly numerically designatedas in FIG. 2.

[0082] The processing algorithm for an RCU 800 is shown in the flowchart of FIG. 8. The flow chart shows only the functions required forunderstanding aspects of the present invention.

[0083] Microcontroller 114 (FIG. 6) scans the input signal of the touchscreen display 122 (FIG. 6). If the viewer presses an object on thescreen at 802, then microcontroller 114 locates and identifies theobject at 804. There are two possibilities; the object can be internalor external at 806. An internal object is an object that is factory-setand independent of the titles provided by service provider 10. Suchinternal objects are used to control the basic functions of multimediadevices, e.g., loudness or channel selection in a manner known in theart and will not be further discussed herein. In such an event, thenon-MPEG data is decoded at 808 with an exit to an internal objectsubroutine at 810.

[0084] An external object is a title that is received from serviceprovider 10. If a displayed title is pressed on the touch-screen displayat 802, then microcontroller 114 creates and transmits a “title requestmessage” at 812, a message is sent to the STB at 814 which stores theobject code of the desired object in memory 118 (FIG. 6) at 816.Additionally, microcontroller 114 sets a bit in a register at 818 (notshown) designated as a “wait-for-object” flag. When this flag is set,then microcontroller 114 “knows” that the viewer has requested an objectfrom the service provider 10 and there is a return to 802.

[0085] When the STB 20 transmits a message to RCU 800, transceiver 108FIG. 6) sends an interrupt signal to microcontroller 114 of RCU 800 at120. Microcontroller 114 checks at 820 if the message contains its ownRCU identification code before proceeding with the decoding process. Ifit is a different identification code, then the message is ignored at822 with a return to 802. If the identification code is correct, thenthe presence of a “not available” message is checked for at 824. Ifthere is such a message, then the “not available” subroutine is enteredat 826.

[0086] If there is no “not available” message, there a check is made fora “switch back” message at 828. If there is such a message, then a“switch back” subroutine is entered at 830. If there is no such message,then the “wait-for-object” flag is checked for at 832. A“wait-for-object” flag that is set means that the RCU is waiting for arequested object, which the object code has been stored in memory 118 ofRCU 800. Microcontroller 114 compares the received object code with thestored object code at 834 and if they are the same, then microcontroller114 decodes the received object data at 836 using MPEG-4 decoder 116(FIG. 6), and resets the “wait-for-object” flag at 838. The receivedobject is then displayed at 840 on display 122 (FIG. 6) with a return to802.

[0087] On the other hand, if the received object code and the storedobject code were not the same, then microcontroller 114 deals with a newobject, i.e. a new title at 842. If the “wait-for-object” flag was notset at 832, then microcontroller 114 also deals with a new title.Microcontroller 114 decodes the new title at 844, arranges the titlesfor display at 846, and displays the titles at 848 using MPEG-4 decoder116, with a return to 802.

1. Apparatus comprising: means for receiving a data signal comprising adata stream including data encoded in a first format for representing anevent and comprising data encoded in a second format for representingadditional information related to the event, wherein the datarepresenting additional information being embedded in the data streamrepresenting the event; means for extracting the additional informationfrom the data stream of the event; and means for providing theadditional information to at least one predetermined remotecommunication unit.
 2. The apparatus of claim 1 wherein the additionalinformation being provided to more than one remote communication unitand only the predetermined remote communication unit being responsive tothe additional information, and further comprising means for receivingfrom the predetermined remote communication unit a request for furtheradditional information.
 3. The apparatus of claim 2 wherein the dataincluded in the data stream encoded in the first format for representingthe event comprises MPEG-2 encoded data and the data encoded in thesecond format for representing the additional information comprisesMPEG-4 encoded data.
 4. The apparatus of claim 3 wherein the data signalreceiving means receives the data signal comprising the datarepresenting the event and the data representing the additionalinformation via a first communications channel.
 5. The apparatus ofclaim 4 further comprising means for providing the request for furtheradditional information to a service provider via one of either the firstcommunications channel or a second communications channel other than thefirst communications channel.
 6. The apparatus of claim 1 furthercomprising means for decoding the data stream representing the event toproduce decoded data, and means for producing a display of the event inresponse to the decoded data, wherein the additional information relatedto the event comprises external objects of additional information aboutthe event, and a number of the external objects becoming invalid withthe passage of time.
 7. The apparatus of claim 6 wherein the externalobjects being valid as long as the event is displayed on the maindisplay device.
 8. The apparatus of claim 7 wherein the data included inthe data signal and encoded in the second format for representing theexternal objects comprises MPEG-4 encoded data.
 9. A method ofprocessing a signal comprising the steps of: receiving a data signalcomprising a data stream including data encoded in a first format forrepresenting an event and comprising data encoded in a second format forrepresenting additional information related to the event, wherein thedata representing additional information being embedded in the datastream representing the event; extracting the additional informationfrom the data stream of the event; and providing the additionalinformation to a predetermined remote communication unit.
 10. The methodof claim 9 wherein the step of providing the additional informationcomprises providing the additional information to more than one remotecommunication unit and only the predetermined remote communication unitbeing responsive to the additional information, and further comprisingthe step of receiving from the predetermined remote communication unit arequest for further additional information.
 11. The method of claim 10wherein the data included in the data stream encoded in the first formatfor representing the event comprises MPEG-2 encoded data and the dataencoded in the second format for representing the additional informationcomprises MPEG-4 encoded data.
 12. The method of claim 11 wherein thestep of receiving the data signal comprises receiving the data signalcomprising the data representing the event and the data representing theadditional information via a first communications channel.
 13. Themethod of claim 12 further comprising the step of providing the requestfor further additional information to a service provider via one ofeither the first communications channel or a second communicationschannel other than the first communications channel.
 14. The method ofclaim 9 further comprising the steps of decoding the data streamrepresenting the event to produce decoded data, and producing a displayof the event in response to the decoded data, wherein the additionalinformation related to the event comprises external objects ofadditional information about the event, and a number of the externalobjects becoming invalid with the passage of time.
 15. The method ofclaim 14 wherein the external objects being valid as long as the eventis displayed on the main display device.
 16. The apparatus of claim 15wherein the data included in the data signal and encoded in the secondformat for representing the external objects comprises MPEG-4 encodeddata.